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Abstract 

The  statistics  of  natural  backgrounds  extracted  from  an  Airborne  Visible  and  Infrared  Imaging  Spectrometer  (AVIRIS) 
hyperspectral  datacube  collected  over  Fort  AP  Hill,  VA,  were  used  to  demonstrate  the  effects  of  the  two  atmospheric 
components  of  a  statistical  end-to-end  performance  prediction  model.  New  capabilities  in  MODTRAN™5  were  used  to 
generate  coefficients  for  linear  transformations  used  in  the  atmospheric  transmission  and  compensation  components  of  a 
typical  end-to-end  model.  Model  radiance  statistics,  calculated  using  reflectance  data,  is  found  to  be  similar  to  the 
original  AVIRIS  radiance  data.  Moreover,  if  identical  atmospheric  conditions  are  applied  in  the  atmospheric 
transmission  and  in  the  atmospheric  compensation  model  components  and  the  effects  of  sensor  noise  are  disregarded,  the 
resulting  reflectance  statistics  are  identical  to  the  original  reflectance  statistics. 

Keywords:  Hyperspectral,  AVIRIS,  Atmospheric  Compensation,  MODTRAN,  spectral  variability 

1 .  Introduction 

The  reflectance  spectra  of  natural  backgrounds  are  often  assumed  to  be  drawn  from  a  multivariate  normal  population. 
However,  recent  work  has  shown  that  even  for  background  populations  carefully  screened  using  clustering  algorithms, 
statistical  descriptions  of  hyperspectral  data  exhibit  long-tail  behavior  that  cannot  be  accounted  for  by  normal 
distribution  models1.  Instead,  statistical  properties  of  natural  hyperspectral  backgrounds  have  been  effectively  modeled 
with  distinct  combinations  of  Gaussian  body  and  /-elliptically  contoured  distributions,  representing  the  heaviness  of  the 
tails1. 

Realistic  probability  density  functions  are  essential  for  accurate  target  detection  prediction  capabilities  and  image 
classification  algorithms1,2,3.  The  design  and  performance  evaluation  of  target  detection  algorithms  for  hyperspectral 
imaging  sensors  depend  on  the  characteristics  of  target  and  background  distributions.  In  general,  the  target  spectral 
signatures  are  obtained  from  spectral  libraries  and  the  background  statistics  are  estimated  from  the  available 
hyperspectral  data.  For  performance  prediction,  statistical  models  for  both  the  spectral  variability  of  the  involved  targets 
and  backgrounds  are  needed. 

Performance  prediction  algorithms  that  rely  on  statistics  derived  from  hyperspectral  scenes4,5,  rather  than  physics-based 
algorithms6,  apply  linear  transformations  to  model  sensor  noise4,5  and  atmospheric  conditions5  prior  to  performance 
prediction  algorithms.  Input  first  and  second  order  statistics,  the  mean  and  covariance  matrix,  can  be  extracted  from 
hyperspectral  imagery  to  describe  the  material  variability.  Figure  1  illustrates  the  processing  flow  for  an  end-to-end 
model.  In  this  model,  input  reflectance  statistics  are  converted  to  at-sensor  radiance  using  a  radiative  transport  model, 
such  as  MODTRAN™ 7  .  Sensor  noise  can  be  added  to  the  resulting  radiance  statistics,  followed  by  removal  of  the 
atmospheric  contaminants  to  simulate  a  real  world  processing  sequence.  The  goal  of  this  paper  is  to  examine  the  effects 
of  the  atmospheric  transmission/compensation  doublet  on  the  spectral  variability  of  natural  backgrounds.  Specifically, 
statistics  derived  from  Airborne  Visible  and  Infrared  Imaging  Spectrometer  (AVIRIS)  imagery  will  be  used  to  illustrate 
the  effects  of  atmospheric  transmission  and  compensation. 

"marv  .glennon. ctrfq)hanscom.af.mil:  phone  781-377-7868 


20080512157 


2.  Methodology 


Following  the  approach  used  by  the  FLAASH  algorithm9,  the  relationship  between  model  at-sensor  radiance,  L(X),  and 
pixel  surface  reflectance,  p(X.),  is  given  by: 


AjX)p{X)  B(X)pa(X) 

0-P.(W))  (1-/>.CW)) 


+  Lp  U) 


(1) 


where  X  represents  the  central  wavelengths  of  the  discrete  spectral  channels  as  measured  by  the  hyper  spectral  sensor, 
p(X)  is  the  pixel  reflectance,  pa(X)  ts  a  spatially  averaged  reflectance  of  an  area  surrounding  the  pixel  of  interest  (used  to 
correct  for  effects  of  light  scattered  from  the  scene  into  the  field  of  view  of  the  senor,  e.g.,  the  adjacency  effect),  S(X)  is 
the  spherical  albedo,  and  LP(X)  is  the  backscattered  radiance.  The  pixel  reflectance  is  assumed  to  be  Lambertian. 

Atmospheric  quantities,  A(X)  and  B(X),  are  the  product  of  the  top-of-atmosphere  solar  downward  horizontal  flux,  total 
sun-to-ground  transmittance,  and  cosine  of  the  solar  zenith  angle.  Furthermore,  A(X)  also  includes  the  direct 
transmission  from  observer  to  sun  while  B(X),  includes  the  medium  embedded  diffuse  transmittance.  Thus,  the  A(X)  and 
B(X)  terms  are  surface  independent  coefficients  that  vaiy  only  with  atmospheric  and  geometric  conditions.  The  first 
term  in  equation  (1)  corresponds  to  the  radiance  reaching  the  surface  (from  both  sky -shine  and  direct  solar  illumination) 
that  is  backscattered  directly  into  the  sensor,  while  the  second  term  corresponds  to  the  radiance  from  the  surface  that  is 
re-scattered  by  the  atmosphere  into  the  sensor  (Figure  2). 

With  the  release  of  MODTRAN™57, 8S  the  atmospheric  contributions  to  at-sensor  radiance  (A,  B ,  Lp  and  S)  in  equation 
(1)  can  be  calculated  with  a  single  MODTRAN  run  using  the  new  DISORT  option  for  atmospheric  correction.  This 
option  exploits  the  reciprocity  principle  of  radiation  transport  theory7, 8.  Equation  1  shows  that  the  conversion  from 
pixel  reflectance  to  at-sensor  radiance  is  a  linear  function,  dependant  upon  the  spectrum  of  the  pixel  of  interest,  the 
spatially  averaged  background  spectrum  to  account  for  the  adjacency  effect  and  the  atmospheric  conditions,  defined  by 
the  visibility  (or  aerosol  content),  water  column  density,  scene  and  sensor  geometry.  Thus  the  transformation  from  pixel 
reflectance  to  at-sensor-radiance  can  be  expressed  using  a  simple  linear  equation  given  by: 

L(A)  =  c(A)p(A)  +  cJ(A)  (2) 

where  vectors  c(X)  and  d(X)  depend  only  on  atmospheric  vectors,  A(X),  B(X),  and  S(X)  calculated  using  MODTRAN™, 
and  the  spatially  averaged  reflectance,  pa(X): 

c(A)= - ^ -  (3) 

o-p.wsw) 


and 


d(X)  = 
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Spectral  library  statistics,  include  (ihe  material  mean  and  covariance  matrix,  are  estimated  from  the  datacube. 
reflectance,  p(X) ,  is  given  by: 


(5) 


The  mean 


and,  the  covariance,  T,  is  given  by : 


(6) 


r = ~2-(a  W  -  p(VXp,W  -  pa)) 

The  Mahanalobis  distance,  A2 ,  given  by! 

a2  =(/KA)-p(> trvw-M)  (?) 

was  also  estimated  from  the  data.  The  probability  of  exceeding  the  Mahanalobis  distance  was  then  determined 
empirically  and  used  as  a  means  to  compare  statistical  distributions. 


Of  more  practical  importance  to  statistical  performance  prediction  models,  the  reflectance  covariance  matrix,  can  be 
transformed  to  radiance  statistics  using  the  relationship: 


where 


1  radiance 


=  cr, 


Re  flee  tan  cey 


C  =  diag{c(Xi),....,c(A„)} 


(8) 

(9) 


The  correlation  coefficient  matrix  (R)  is  given  by: 


Note  that  R  is  independent  of  atmospheric  state.  R  is  of  importance  because  the  probability  density  function  for  a 
multivariate  /-ECD,  used  to  model  the  statistical  variability  of  hyperspectral  data,  is  a  function  of  the  material 
reflectance,  the  correlation  coefficient  matrix  and  the  number  of  degrees  of  freedom2.  For  the  case  where  TB  *  ,  the 

target  can  be  modeled  with  a  Gaussian  model  because  the  probability  of  detection  depends  upon  the  body  of  the  target 
distribution  and  not  the  tails2,3. 


3.  Data  Description 

We  use  hyperspectral  data  collected  with  the  AVIRIS  sensor  over  Fort  AP  Hill  in  September,  2001  (Figure  3).  Fort  AP 
Hill  is  typical  of  a  fairly  rural  northeast  forest  with  both  deciduous  and  coniferous  tree  stands.  The  AVIRIS  instrument 
is  sensitive  from  375  to  2500  nm  in  224  spectral  channels  with  a  nominal  bandwidth  of  about  lOnm.  Data  were 
calibrated  to  at-sensor  radiance  and  distortions  due  to  aircraft  motion  were  removed  by  JPL.  The  sensor  was  flown  on 
the  Twin  Otter  aircraft  in  low-altitude  nadir-looking  configuration,  providing  a  GSD  of  approximately  3.5m. 

Calibrated  radiance  was  atmospherically  corrected  to  reflectance  using  the  MODTRAN-based  Fast  Line  of  Sight 
Analysis  of  Atmospheric  Spectral  Hypercubes  (FLAASH)9.  Water  vapor  was  retrieved  from  the  radiance  cube  using 
the  940  nm  water  vapor  absorption  region  separately  for  each  pixel  in  the  scene.  FLAASH  calculated  the  average  water 
vapor  at  1400  atm-cm.  The  scene-averaged  visibility  or  aerosol  overburden  is  calculated  using  a  reflectance  ratio-based 
algorithm  based  upon  an  empirical  model  by  Kaufman10.  For  the  Fort  AP  Hill  scene,  the  average  visibility  was 
calculated  at  122  km. 


The  spectral  library  data  used  for  this  study  was  extracted  from  the  FLAASH  reflectance  cube  using  spectrally  distinct 
classes  reported  by  Cipar  and  others11.  The  unsupervised  classification  algorithm11  determines  cluster  centers  with  a 
modified  migrating  means  technique  to  separate  the  dense  parts  of  the  hyperspectral  data  cloud.  A  total  of  1 9  classes 
were  reported  by  Cipar  and  others11.  Two  distinct  classes  were  chosen  for  this  experiment  with  sufficient  pixels  to 
provide  a  good  estimate  of  the  mean  and  covariance  matrix.  A  total  of  13,808  pixels  lie  within  class  6  and  are 
highlighted  in  blue  in  Figure  3b.  From  visual  inspection  and  personal  site  inspections,  class  6  appears  to  consist  mainly 
of  grass  and  other  low-lying  vegetation.  Class  #2  (in  red)  was  identified11  as  a  deciduous  class  and  consists  of  32,593 
pixels.  Spectral  channels  with  low  signal  to  noise  ratios  and  those  near  the  strong  water  absorption  features  near  1350 
and  1 900  nm  were  removed  from  further  calculations,  leaving  155  “good”  channels  for  analysis. 


4.  Results 

Scene  and  atmospheric  conditions  used  as  input  to  MODTRAN  to  calculate  the  atmospheric  constants  in  Equation  1  (A, 
B ,  S',  and  Lp  )  are  listed  in  Table  1 .  The  visibility  and  scene-averaged  water  column  were  retrieved  by  FLAASH  during 
the  original  atmospheric  compensation  step.  For  all  calculations,  the  spatially  averaged  reflectance,  pfl  ,  in  equation  (1), 
was  defined  as  the  reflectance  mean  of  the  entire  scene.  Note  that  FLAASH  both  retrieves  the  water  column  totals  and 
calculates  the  spatially  averaged  reflectance,  separately  for  each  pixel  in  the  scene.  In  contrast,  the  MODTRAN 
model  parameters  used  in  this  study,  including  water  content  and  pfl,  are  identical  for  each  pixel. 

We  first  apply  equation  (1)  to  each  pixel  in  the  atmospherically  corrected  reflectance  datacube  using  the  155  ‘good’ 
spectral  channels  and  the  model  atmosphere  (Table  1)  to  produce  a  model  radiance  datacube,  L .  Pixel  spectra  were 

extracted  from  the  model  radiance  cube,  L  ,  and  the  original  AVIRIS  radiance  cube,  L,  for  each  class  and  the  mean  and 
covariance  estimated.  Finally,  the  Mahanalobis  distance  is  computed.  The  statistics  extracted  from  the  model 

radiance,  L  ,  were  compared  to  that  of  the  original  AVIRIS  calibrated  radiance,  L. 

The  class  means  calculated  using  the  original  AVIRIS  radiance  dataset  and  the  model  radiance  cube  are  very  similar 
(Figure  4)  for  both  classes.  Differences  between  the  two  means  for  each  class  are  generally  less  than  2%  and  are  nearly 
identical.  Larger  differences  are  observed  at  discrete  bands  associated  with  atmospheric  absorption  regions  at  940,  1 1 30 
nm  and  at  the  edges  of  the  opaque  atmospheric  windows  near  1460  and  1750  nm.  Given  that  the  FLAASH  algorithm 
retrieved  water  vapor  amounts  at  each  pixel  and  that  equation  (1)  uses  a  single  water  vapor,  these  variations  are  not 
surprising.  Within  each  class,  variations  in  water  vapor  are  predicted  by  the  FLAASH  algorithm.  For  example,  for  the 
pixels  within  the  grass  class,  the  mean  water  vapor  was  estimated  at  1448  atm-cm  with  a  standard  deviation  of  48  atm- 
cm,  slightly  higher  than  the  estimate  for  the  whole  cube.  In  addition,  both  methodologies  assume  a  uniform  visibility 
across  the  scene  and  a  standard  mid-latitude  summer  aerosol  model.  It  is  conceivable  that  aerosols  may  change  across  a 
scene.  Furthermore,  more  sophisticated  aerosol  models,  such  as  a  power  law  distribution,  may  provide  more  accurate  at- 
sensor  radiance  estimates12. 

Equation  10  predicts  that  if  the  atmospheric  model  was  exactly  correct  for  this  scene,  then  the  correlation  matrix  for  the 
model  radiance  data  and  the  original  AVIRIS  data  cube  should  be  nearly  identical.  In  reality,  the  atmospheric  model  in 
(1)  is  a  simplified  description  of  the  actual  atmospheric  parameters,  such  as  effects  of  aerosol  and  water  vapor. 
Nevertheless,  differences  between  model  and  actual  correlation  coefficients  are  less  than  5%  for  over  93%  of  the  matrix. 
In  all  cases,  the  larger  correlation  coefficient  differences  are  associated  with  spectral  channels  in  the  water  vapor 
absorption  regions. 

The  empirical  exceedence  distributions  of  Mahanalobis  distance  for  each  class  are  used  as  another  tool  to  evaluate 
differences  between  the  model  radiance  and  the  original  AVIRIS  radiance.  This  metric  has  been  used  to  characterize 
the  statistics  of  natural  backgrounds  imaged  by  hyperspectral  sensors1.  Similar  to  previous  studies1,  the  distributions  of 
Mahanalobis  distance  for  both  classes  do  not  follow  the  normal  distribution  (Figure  5).  Instead,  the  heavy  tails 
exhibited  by  the  grass  and  deciduous  classes  are  very  similar  to  the  original  data.  The  original  radiance  data,  plotted  as 
dotted  curves,  exhibit  higher  Mahanalobis  distances  for  lower  probabilities  of  exceedence  than  does  the  model  data, 
suggesting  that  the  variability  in  the  model  data  is  slightly  reduced  relative  to  the  original  dataset.  Although  no  effort 
was  made  to  characterize  the  heavy  tails  for  either  distribution,  the  differences  are  predicted  to  be  small. 


In  the  above  discussion,  pixels  within  each  class  are  used  to  calculate  the  means  and  the  covariances  for  both  the  model 
and  original  radiances.  For  statistical  models,  however,  the  atmosphere  is  propagated  directly  onto  material  statistics 
(mean  and  covariance)  extracted  from  a  library.  For  each  class,  the  difference  between  mean  radiances  and  covariances 
calculated  using  the  two  sources  are  less  than  0.1% 

In  the  end-to-end  performance  tool  depicted  in  Figure  1 ,  the  atmospheric  compensation  step  removes  the  contamination 
of  the  atmosphere  from  model  radiance  statistics  prior  to  performance  statistics  calculations.  If  identical  atmospheric 
parameters  are  used  to  compensate  the  model  radiance  using  the  inverse  of  equation  3,  then  the  resulting  reflectance 
means  and  covariance  statistics  will  be  identical  to  the  original  input  statistics,  excluding  the  effects  of  sensor  noise.  If 
the  errors  associated  with  atmospheric  compensation  are  to  be  modeled,  then  different  atmospheric  conditions  must  be 
used  to  calculate  the  constants  in  equation  1  for  use  in  the  atmospheric  compensation  step. 

The  differences  between  the  AVIRIS  radiance  and  model  radiance  class  means  (Figure  4)  can  be  considered  an  upper 
bound  to  errors  due  to  atmospheric  compensation  and  are  generally  less  than  5%.  The  AVIRIS  data  used  in  this  study 
was  collected  under  nearly  ideal  conditions  for  hyperspectral  imagery;  clear  to  moderate  aerosol/haze,  low  to  moderate 
water  vapor  and  nadir  viewing  geometries.  Surface  reflectance  produced  with  the  FLAASH  algorithm  has  been 
compared  with  high  accuracy13  to  ground  spectral  measurements  under  similar  conditions.  More  stressing  scenarios, 
such  as  scenes  collected  with  an  off-nadir  geometry,  scenes  with  higher  water  content  and/or  lower  visibilities  (heavy 
aerosol  conditions)  may  exhibit  larger  errors  when  the  linear  algorithms  for  atmospheric  transmission  and  compensation 
are  applied.  Further  analysis  of  hyperspectral  datacubes  collected  with  varying  conditions  is  needed  to  understand  the 
limitations  of  this  formulation  of  the  radiance  equation. 


Table  1 


Scene  Parameters 

Original  Scenario 

Latitude,  Longitude 

38.125  N.  77.325  W 

Atmospheric  Model 

Mid-Latitude  Summer 

Sensor  Altitude 

3.55  km 

Time  (UTC) 

15.40 

Visibility  (km) 

122 

Water  Column 

1400  atm-cm 

Summary 

The  statistics  of  natural  backgrounds  extracted  from  an  AVIRIS  hyperspectral  datacube  collected  over  Fort  AP  Hill,  VA, 
were  used  to  demonstrate  the  effects  of  the  two  atmospheric  components  (transmission  and  compensation)  of  a  statistical 
end-to-end  performance  prediction  model.  First,  new  capabilities  in  MODTRAN™5  were  used  to  generate  coefficients 
for  the  linear  transformation  used  in  the  atmospheric  transmission  component  of  a  typical  end-to-end  performance 
prediction  model.  Statistics  extracted  from  the  model  radiance  cube  were  compared  to  that  of  the  original  AVIRIS 
radiance  cube.  Class  means  differed  by  less  than  3-5%  in  regions  outside  of  the  atmospheric  absorption  bands. 
Distributions  of  Mahanalobis  distance  for  both  datasets  are  also  very  similar.  Furthermore,  for  the  scene  used  in  this 
study,  if  identical  atmospheric  conditions  are  used  in  the  linear  transformation  to  reflectance,  the  resulting  reflectance 
statistics  are  identical  to  the  original  reflectance  statistics. 
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Figure  1.  Procedures  for  end-to-end  performance  prediction  models 


Figure  2.  Illustration  of  contributions  to  the  target  pixel  upwelling  radiance. 
The  yellow  line  segment  corresponds  to  the  radiance  backscattered  directly  into 
the  sensor  from  the  pixel  of  interest,  the  blue  arrow  corresponds  to  radiance 
reflected  by  the  earth’s  surface  surrounding  the  pixel  and  backscattered  into  the 
sensor.  The  green  arrow  corresponds  to  radiance  backscattered  by  the 
atmosphere  without  reaching  the  surface. 


Figure  5.  The  distribution  of  Mahanalobis  distance  calculated  for  the  spectral  classes  shown  in  Figure  3.  The  solid  curves  represent 
the  distribution  using  the  model  radiance  and  the  dotted  curves  use  the  original  AVIRIS  radiance  cube.  Red  and  blue  curves  are 
associated  with  class  2  (deciduous)  and  class  6  (grass),  respectively.  The  abscissa  was  truncated  at  a  Mahanalobis  distance  of  1600 
for  clarity.  The  black  curve  represents  the  normal  distribution  model. 


